Unify input output by jo-mueller · Pull Request #117 · ome/ngff-spec

jo-mueller · 2026-03-26T14:34:48Z

Fixes ome/ngff#480
Fixes ome/ngff#437
Relevant for ome/ngff#360

Description

In previous versions of the spec, we had used a mix of contentions when it came to specifying the inputs and outputs of coordinate transformations:

In the scene context, input and output had to be an object with fields name and path.
For the multiscale transformations inside the multiscales metadata (multiscales > datasets > coordinateTransformations), only a string was allowed, which had to correspond to the path of the respective dataset entry.
For the additional transforms next to the multiscales metadata (multiscales > coordinateTransformations), a string was required, which had to correspond top the name of a coordinate system in the same metadata document.

Following the original suggestion of @dstansby at ome/ngff#437, this is unified into a common syntax in this PR. It also has adjustments for schemas, examples and CI tests.

cc @will-moore @dstansby @clbarnes @bogovicj @lorenzocerrone @jluethi @m-albert @thewtex

…name/path

github-actions · 2026-03-26T14:35:02Z

Automated Review URLs

Readthedocs

dstansby

Some comments, but 👍 this looks great overall

index.md

Co-authored-by: David Stansby <dstansby@gmail.com>

…sforms

Co-authored-by: David Stansby <dstansby@gmail.com>

…raints

jo-mueller · 2026-04-01T07:16:01Z

Regarding a discussion over at ome/ngff#339, I think it may be better to turn the requirement of the containing multiscales having to declare a coordinate transform to a contained label image into a SHOULD or even a MAY. Otherwise, doing the following thing essentially invalidates the parent multiscales image:

You write a multiscales image IMG, complete with all metadata
You run a segmentation workflow and store the result as another multiscales under image/labels/some_segmentation
In the current state, if you wouldn't write a coordinate transform into IMG's metadata, it would now be invalid, which I think would be highly problematic.

Maybe the following would be better:

Requiring that multiscales under labels/wherever MUST only have one coordinate system.
If no coordinate transformation is written under IMG, it is implicit for "native coordinate system of IMG and label image under labels/whereever are linked by an identity transform"

will-moore · 2026-04-01T08:24:20Z

index.md

+| Context | `input` | `output` |
+|---------|---------|----------|
+| **multiscales > datasets** | `{ "path": "<dataset_path>" }` | `{ "name": "intrinsic" }`|
+| **multiscales > coordinateTransformations** | `{ "name": "intrinsic" }` | `{ "name": "output" }` <br> or <br> `{ "name": "intrinsic", "path": "labels/labels_path" }` |


I would expect any labels to be input rather than output, as the labels would be the lowest-level "leaves" of the tree that has it's apex as the "scene" at the top, with the multscale images "intrinsic" coordinateSystem in the middle.

Can do, but that means that we'd need to weaken the requirement in the multiscales section further down, where it says:

If applications require additional transformations,
each multiscales object MAY contain the field coordinateTransformations,
describing transformations that are applied to all resolution levels in the same manner.
The values of both input and output fields MUST be an object with fields name and path that satisfy:

The value of input MUST be the "intrinsic" coordinate system, referenced by name.
The path field of input SHOULD be omitted.

The correct replacement of that statement would then be:

The value of either input or output MUST be the "intrinsic" coordinate system, referenced by name. The respective path field SHOULD be omitted.

I think that the multiscales requirements do need to be relaxed, both to allow labels as inputs, but also because you might have intrinsic -transformed-to-> deskewed -transformed-to-> rotated then the rotation transform would not have input as intrinsic. We'd want to allow that, right?

I'm still not clear about the "intrinsic" coordinateSystem rules. Is this a term that refers to the coordinate system that behaves as the intrinsic system (all datasets output to intrinsic), or is it the case that every multiscales image MUST have a coordinateSystem that has "name": "intrinsic"?
And should viewers always attempt to show the "intrinsic" coordinateSystem? If I have e.g. the intrinsic -transformed-to-> deskewed then I may/probably want any viewer to show the deskewed coordinateSystem?

you might have intrinsic -transformed-to-> deskewed -transformed-to-> rotated

Nope, currently not allowed. all transforms under multiscales -> coordinateTransformations must be linked to the same coordinate system (the "intrinsic" coordinate system) to limit graph complexity. So to do what you are describing you would have to choose a sequence of intrisinc -(affine + rotate)-> rotated.

About the "intrinsic" CS, I'll add a clarifying remark further up. Essentially: The "intrinsic"/"native"/"physical" coordinate system is the one that all multiscale transforms out put to.

OK, understood. This all seems consistent now. Just unsure about "[the intrinsic coodindate system] should be used for viewing and processing unless a use case dictates otherwise".
If I'm implementing a viewer, how do I know whether to show the "intrinsic" coordinateSystem or some other (when just viewing the image, rather than the whole "scene")?

Yeah, this definitely needs a clarifying remark further up as to what the "intrinsic" coordinate system denotes.

Yep - I was reminded that Davis asked about that too at #118 (comment)

Yeah, this definitely needs a clarifying remark further up as to what the "intrinsic" coordinate system denotes.

Hey :) I was thinking about that...

I'd agree with Davis' #118 (comment) that defining the intrinsic coordinate system as the "native physical coordinate system" is a bit misleading. I think the intention behind "It should be used for viewing" is important, but in my view it is already captured by the description of the transformations which have the intrinsic coordinate system as output (in the datasets objects):

In these cases, the scale transformation specifies the pixel size in physical units or time duration.

Probably that's all an implementation could know for sure about physical coordinates.

With this and the discussion in #118 (comment) in mind, how about having the definition of the intrinsic coordinate system more descriptive:

To both initialize the coordinates of a multiscale image and to define the relative scaling factors between resolution levels, multiscale images have a special coordinate system, the "intrinsic" coordinate system. It is the coordinate system that serves as the common output coordinate system for all transformations specified for the objects in the datasets field of a multiscale object.

index.md

will-moore · 2026-04-01T10:43:05Z

I think we need to review some of the existing labels statements, since we now MUST refer to labels with the input/output of a transform in the parent image.
E.g.
In the tree layout

# The labels group is a container which holds an array of labels to make the objects easily discoverable
# All labels will be listed in zarr.json e.g. { "labels": [ "original/0" ] }
# Multiscale, labeled image. The name is unimportant but is registered in the "labels" group above.

Although the layout rules are unchanged, these statements are incomplete as the layout is not the only way to discover labels now, and labels are not only listed in the labels/zarr.json.

Also: "Within the multiscales object, the JSON array associated with the datasets key MUST have the same number of entries (scale levels) as the original unlabeled image".

This came up at https://forum.image.sc/t/ome-zarrpari-an-ome-zarr-napari-widget/119772/9 as being overly strict. Especially if we now allow scale/translation to map labels to the parent image, this seems outdated.

jo-mueller · 2026-04-01T10:53:35Z

I think we need to review some of the existing labels statements, since we now MUST refer to labels with the input/output of a transform in the parent image.

Yes, that's clearly too strict 🙈

will-moore · 2026-04-01T11:58:06Z

I just thought of another issue with specifying identity, scale, translation transforms between labels and image coordinateSystems: All of those transforms will preserve all axes but I think most people assume that labels won't have a "channel" axis? There's been a proposal (somewhere) to say that labels shouldn't have a channel axis.

The spec refers to a label image as "(usually having the same dimensions and coordinate transformations)" as the parent image. But it doesn't say that it MUST have the same axes.

# Each dimension of the label should be either the same as the
# corresponding dimension of the image, or `1` if that dimension of the label
# is irrelevant.

So I think we need to clarify whether label images can omit the Channel axis. If label images don't have a channel axis, then how do we handle that with a transform that goes from coordinateSystem

labels (no channel)

{
    "name" : "my_label",
    "axes" : [
        {"name": "z", "type": "space", "unit": "micrometer"},
        {"name": "y", "type": "space", "unit": "micrometer"},
        {"name": "x", "type": "space", "unit": "micrometer"}
    ]
}

to image (with channel)

{
    "name" : "intrinsic",
    "axes" : [
        {"name": "c", "type": "channel"},
        {"name": "z", "type": "space", "unit": "micrometer"},
        {"name": "y", "type": "space", "unit": "micrometer"},
        {"name": "x", "type": "space", "unit": "micrometer"}
    ]
}

mkitti · 2026-04-01T19:16:25Z

If I had two arrays with different or missing dimensions I would expect some sort of broadcasting to apply:
https://numpy.org/doc/stable/user/basics.broadcasting.html
https://blog.glcs.io/broadcasting

m-albert · 2026-04-02T00:58:45Z

It might not be at the core of this PR, but it's not entirely clear to me from the spec text whether the "intrinsic coordinate system" needs to actually have the name "intrinsic"? Or can it be any other name and we just call this specific coordinate system the intrinsic coordinate system? In case I didn't miss it, it could make sense to clarify that.

jo-mueller added 6 commits March 26, 2026 15:13

schemas: introduced central InputOutput object definition

d4c1c2c

tests: use correct InputOutput everywhere

d453396

docs: update examples for correct InputOutput syntax

82f6c8f

chore: Updated version to 0.6.dev4 everywhere

0ab6faf

specification: Unifiy input/output of transformations to object with …

34fa54c

…name/path

chore: update version history

d883e55

jo-mueller added the enhancement New feature or request label Mar 26, 2026

jo-mueller added 2 commits March 26, 2026 17:29

specification: improve clarity

38f9334

chore: fix typos

85056a3

jo-mueller marked this pull request as ready for review March 26, 2026 16:31

jo-mueller changed the title ~~WIP: Unify input output~~ Unify input output Mar 26, 2026

dstansby reviewed Mar 27, 2026

View reviewed changes

jo-mueller and others added 9 commits March 27, 2026 17:26

Update index.md

9849a4c

Co-authored-by: David Stansby <dstansby@gmail.com>

Update index.md

98a4d56

Co-authored-by: David Stansby <dstansby@gmail.com>

specification: Clarify constraintsfor input/output in additional tran…

9c7dae2

…sforms

Update index.md

3076fb7

Co-authored-by: David Stansby <dstansby@gmail.com>

Update index.md

80703fc

Co-authored-by: David Stansby <dstansby@gmail.com>

Update index.md

8c0674c

Co-authored-by: David Stansby <dstansby@gmail.com>

Update index.md

b0108b2

Co-authored-by: David Stansby <dstansby@gmail.com>

chore: Clearer on language

2ab8a28

chore: Improve readability on multiscale transform input/output const…

0bd9da6

…raints

lubianat mentioned this pull request Mar 30, 2026

NGFF PR Review (2026-04-01) German-BioImaging/incubator#60

Closed

clbarnes approved these changes Mar 31, 2026

View reviewed changes

clbarnes mentioned this pull request Mar 31, 2026

specification: Clarify coordinates and displacements transformations #108

Open

jo-mueller requested a review from dstansby March 31, 2026 15:38

This was referenced Mar 31, 2026

Replace arrayCoordinateSystem with explanation on how to express dimensionless transforms in pixel coordinates #118

Open

No way to distinguish image and labels-image groups in 0.5 ome/ngff#339

Open

will-moore reviewed Apr 1, 2026

View reviewed changes

index.md Outdated Show resolved Hide resolved

specification: allow sequence in labels transforms

bd5175d

Merge remote-tracking branch 'upstream/main' into unify-input-output

8a32f83

Conversation

jo-mueller commented Mar 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Uh oh!

github-actions bot commented Mar 26, 2026

Automated Review URLs

Uh oh!

dstansby left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jo-mueller commented Apr 1, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

will-moore commented Apr 1, 2026

Uh oh!

jo-mueller commented Apr 1, 2026

Uh oh!

will-moore commented Apr 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mkitti commented Apr 1, 2026

Uh oh!

m-albert commented Apr 2, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

jo-mueller commented Mar 26, 2026 •

edited

Loading

will-moore commented Apr 1, 2026 •

edited

Loading